No-reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics

We present two contributions in this work: (i) a bivariate generalized Gaussian distribution (BGGD) model for the joint distribution of luminance and disparity subband coefficients of natural stereoscopic scenes and (ii) a no-reference (NR) stereo image quality assessment algorithm based on the BGGD model. We first empirically show that a BGGD accurately models the joint distribution of luminance and disparity subband coefficients. We then show that the model parameters form good discriminatory features for NR quality assessment. Additionally, we rely on the previously established result that luminance and disparity subband coefficients of natural stereo scenes are correlated, and show that correlation also forms a good feature for NR quality assessment. These features are computed for both the left and right luminance-disparity pairs in the stereo image and consolidated into one feature vector per stereo pair. This feature set and the stereo pair׳s difference mean opinion score (DMOS) (labels) are used for supervised learning with a support vector machine (SVM). Support vector regression is used to estimate the perceptual quality of a test stereo image pair. The performance of the algorithm is evaluated over popular databases and shown to be competitive with the state-of-the-art no-reference quality assessment algorithms. Further, the strength of the proposed algorithm is demonstrated by its consistently good performance over both symmetric and asymmetric distortion types. Our algorithm is called Stereo QUality Evaluator (StereoQUE)

Quality Aware Generative Adversarial Networks

Generative Adversarial Networks (GANs) have become a very popular tool for implicitly learning high-dimensional probability distributions. Several improvements have been made to the original GAN formulation to address some of its shortcomings like mode collapse, convergence issues, entanglement, poor visual quality etc. While a significant effort has been directed towards improving the visual quality of images generated by GANs, it is rather surprising that objective image quality metrics have neither been employed as cost functions nor as regularizers in GAN objective functions. In this work, we show how a distance metric that is a variant of the Structural SIMilarity (SSIM) index (a popular full-reference image quality assessment algorithm), and a novel quality aware discriminator gradient penalty function that is inspired by the Natural Image Quality Evaluator (NIQE, a popular no-reference image quality assessment algorithm) can each be used as excellent regularizers for GAN objective functions. Specifically, we demonstrate state-of-the-art performance using the Wasserstein GAN gradient penalty (WGAN-GP) framework over CIFAR-10, STL10 and CelebA datasets.Comment: 10 pages, NeurIPS 201

Siamese Cross-Domain Tracker Design for Seamless Tracking of Targets in RGB and Thermal Videos

Multimodal (RGB and thermal) applications are swiftly gaining importance in the computer vision community with advancements in self-driving cars, robotics, Internet of Things, and surveillance applications. Both the modalities have complementary performance depending on illumination constraints. Hence, a judicious combination of both modalities will result in robust RGBT systems capable of all-day all-weather applications. Several studies have been proposed in the literature for integrating the multimodal sensor data for object tracking applications. Most of the proposed networks try to delineate the information into modality-specific and modality shared features and attempt to exploit the modality shared features in enhancing the modality specific information. In this work, we propose a novel perspective to this problem using a Siamese inspired network architecture. We design a custom Siamese cross-domain tracker architecture and fuse it with a mean shift tracker to drastically reduce the computational complexity. We also propose a constant false alarm rate inspired coasting architecture to cater for real-time track loss scenarios. The proposed method presents a complete and robust solution for object tracking across domains with seamless track handover for all-day all-weather operation. The algorithm is successfully implemented on a Jetson-Nano, the smallest graphics processing unit (GPU) board offered by NVIDIA Corporation

Sparsity based stereoscopic image quality assessment

In this work, we present a full-reference stereo image quality assessment algorithm that is based on the sparse representations of luminance images and depth maps. The primary challenge lies in dealing with the sparsity of disparity maps in conjunction with the sparsity of luminance images. Although analysing the sparsity of images is sufficient to bring out the quality of luminance images, the effectiveness of sparsity in quantifying depth quality is yet to be fully understood. We present a full reference Sparsity-based Quality Assessment of Stereo Images (SQASI) that is aimed at this understanding

Multiscale-SSIM Index Based Stereoscopic Image Quality Assessment

Stereoscopic image quality typically depends on two factors: i) the quality of the luminance image perception, and ii) the quality of depth perception. The effect of distortion on luminance perception and depth perception is usually different, even though depth is estimated from luminance images. Therefore, we present a full reference stereoscopic image quality assessment (FRSIQA) algorithm that rates stereoscopic images in proportion to the quality of individual luminance image perception and the quality of depth perception. The luminance and depth quality is obtained by applying the robust Multiscale-SSIM (MS-SSIM) index on both luminance and disparity maps respectively. We propose a novel multi-scale approach for combining the luminance and depth scores from the left and right images into a single quality score per stereo image. We also explained that a small amount of distortion does not significantly affect depth perception. Further, heavy distortion in stereo pairs will result in significant loss of depth perception. Our algorithm performs competitively over standard databases and is called the 3D-MS-SSIM index

Target Tracking in Blind Range of Radars With Deep Learning

Surveillance radars form the first line of defense in border areas. But due to highly uneven terrains, there are pockets of vulnerability for the enemy to move undetected till they are in the blind range of the radar. This class of targets are termed the 'pop up' targets. They pose a serious threat as they can inflict severe damage to life and property. Blind ranges occur by way of design in pulsed radars. To minimize the blind range problem, multistatic radar configuration or dual pulse transmission methods were proposed. Multistatic radar configuration is highly hardware intensive and dual pulse transmission could only reduce the blind range, not eliminate it. In this work we propose, elimination of blind range using deep learning based video tracking for mono static surveillance radars. Since radars operate in deploy and forget mode, visual system must also operate in a similar way for added advantage. Deep Learning paved way for automatic target detection and classification. However, a deep learning architecture is inherently not capable of tracking because of frame to frame independence in processing. To overcome this limitation, we use prior information from past detections to establish frame to frame correlation and predict future positions of target using a method inspired from CFAR in a parallel channel for target tracking. © 2020 Warsaw University of Technology